(19) 



J 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(45) Date of publication and mention 
of the grant of the patent: 
14.04.1999 Bulletin 1999/15 

(21) Application number: 94101650.3 

(22) Date of filing: 03.02.1994 



(n) EP 0 618 728 B1 

EUROPEAN PATENT SPECIFICATION 

(51) lntCI.6; H04N 7/24 



(54) Synchronization of audio/video information 

Synchronisierung von Ton-und Videoinformation 
Synchronisation d'inforrnation audio/video 



(84) Designated Contracting States: 
DE FR GB 

(30) Priority: 26.02.1993 JP 63293/93 
16.03.1993 US 32341 

(43) Date of publication of application: 
05.10.1994 Bulletin 1994/40 

(73) Proprietor: SONY CORPORATION 
Tokyo (JP) 

(72) inventor: Veltman, Markus H., 
c/o Sony Corporation 
Tokyo (JP) 



CD 

CO 
CM 

00 
CO 

o 

LU 



(74) Representative: Melzer, Wolfgang, Dipl.-lng. et at 
Patentanwalte 
Mitscheriich & Partner, 
Sonnenstrasse 33 
80331 Munchen (DE) 



(56) References cited: 
FR-A- 2 648 972 



US-A-5 138 440 



• SIGNAL PROCESSING. IMAGE 

COMMUNICATION, vol.3, no.2/3, June 1991, 
AMSTERDAM NL pages 157 - 165 LEDUC 
'universal VBR videocodecs for ATM networks 
in the Belgian Broadband Experiment' 



Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give 
notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in 
a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 
99(1) European Patent Convention). 



Printed by Jouve, 75001 PARIS (Ffl) 



1 



EP 0 618 728 B1 



2 



Description 

Background 

1- Field of the Invention 

[0001] This invention relates generally to the field of 
digital transmission or storage of audio and video infor- 
mation. More particularly, this invention relates to a tech- 
nique for verifying the timing of multiplexed packetized 
audio and/or video (A/V) information such as ISO's (In- 
ternational Organization for Standardization) MPEG 
(Motion Pictures Expert Group). 

2. Background of the Invention 

[0002] The term "access unit" as used herein means 
either a frame of video data or a batch of audio samples 
or a batch of other data samples. In general, a decoder 
processes it's "access units" and outputs decoded ac- 
cess units at regular intervals. In the case of video, for 
example, this interval is the picture rate, and for audio 
this is a constant integer (equal to the number of audio 
samples in an audio access unit) times the audio sam- 
pling rate. 

[0003] in the case of an ISO standard MPEG system 
application, as defined for example in ISO document 
number 1-11172, a "time stamp" indicatingthe decoding 
time for an access unit, is included in the multiplex syn- 
tax for multiplexed video and audio packets as shown 
in FIGURE 1. These time stamps are included in the 
multiplex syntax, i.e. in packet headers. A time stamp 
indicates the decoding time of the first access unit head- 
er in that packet. Since each packet can include multiple 
access units, each access unit is not associated with a 
time stamp. 

[0004] Although multiplex applications use multiple 
decoders, ISO's MPEG 1 system standard can also be 
applied in applications that only have one decoder. In a 
fixed bit rate (e.g. audio) application, perfect clocks do 
not exist, therefore the digital storage media (or trans- 
mission) bit rate and (depending on the sampling clock 
frequency error) the decoder input bit rate vary. In such 
a system, the decoder generally reads one access unit 
at a time. Furthermore, due to the differences in the 
clocks, the transfer bit rate and the decoder input bit rate 
do not identically match. A buffer can be used to com- 
pensate for these differences. In the case of video data, 
consecutive access units (i.e. frames) are compressed, 
which depending on the picture content, results in vari- 
able length compressed (VLC) access units. Conse- 
quently the video decoder input bit rate has large varia- 
tions and a relatively large buffer is used. 
[0005] However, even if a buffer is used, ideal bit rates 
generally do not exist and therefore "buffer errors" (and 
even buffer overflow or underflow) can occur. Two meth- 
ods are generally used to prevent buffer overflow and 
underflow. With one method (called Digital Storage Me- 



dia slave) the transfer rate (i.e. buffer input rate) is con- 
trolled. The other method (called decoder slave) is real- 
ized by controlling the buffer output data rate. I n the case 
of video this can be done by adjusting the frame rate. In 

5 the case of audio, this is done by adjusting the sampling 
rate. The buffer output data rate is thus adjusted. An- 
other decoder slave method skips or repeats access 
units in order to control the buffer output data rate. 
[0006] Adjustments of the decoder rate and adjust- 

10 ments of the transfer bit rate are restricted by charac- 
teristics of the peripheral hardware. Therefore if the buff- 
er error (i.e. deviation from the ideal buffer fullness) is 
too large, the appropriate control can become difficult 
or impossible. When starting playback, a large buffer er- 

15 ror can sometimes occur. Therefore generally the de- 
coder starts playback after an appropriate start up delay 
in order to reduce the initial buffer error. 
[0007] In the MPEG 1 system standard : fields have 
been included in the multiplex syntax which can be used 

20 to control the decoder or the transfer rate. In the pack 
header a value called SCR (System Clock Reference) 
can be used to control the transfer data rate. Time 
stamps in the video packet header can be used to con- 
trol the frame rate, and time stamps in the audio packet 

25 header can be used to control the sampling rate. SCR 
indicates the time when (the first part of the) packet data 
enters the decoder buffer and time stamps indicate 
when a certain access unit in the packet data leaves the 
decoder buffer. 

30 [0008] Both SCR and time stamps are absolute val- 
ues of a clock that increments continuously at a rate of 
90 KHz. Therefore the difference between the first read 
SCR and the first read time stamp can be used as a start 
up delay. 

35 [0009] Unfortunately, (except for the first audio and 
video time stamps for initial start up) it is difficult to use 
consecutive time stamps. The problem is that after de- 
multiplexing time stamps are separated from their relat- 
ed access units. I.e., since the decoding system's de- 

40 multiplex switch separates time stamps from the packet 
data and then stores the packet data (without their time 
stamps) in the respective buffers, it becomes difficult to 
keep track of which time stamp belongs to which access 
unit header. For example: a decoding system processes 

45 a certain MPEG multiplex stream. The first SCR (Sys- 
tem Clock Reference) is detected, and the system uses 
this value to initialize a local (90KHz) clock. From now 
on this clock increments automatically at a 90KHz rate. 
Then the system detects the first video DTS. It indicates 

50 (with a 90KHz clock value) the decoding time for the first 
following picture header. However generally there is a 
significant delay before this picture header should be 
decoded (due to the buffer before the video decoder), 
and therefore the time stamp must also be delayed or 

55 buffered before it can be used. Furthermore, before the 
first picture is decoded : several more video packets with 
time stamps can arrive at the demultiplexer switch and 
these time stamps should also be delayed or buffered 
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somewhere before they can be used. 
[0010] With the adopted definition for time stamps, it 
is a significant shortcoming of the MPEG standard that 
the time stamps were not included in the video and audio 
syntax specifications. 5 
[0011] The problem is that the time stamps are in the 
"wrong" layer. This might not be so bad if all access unit 
headers had time stamps. However, this is not the case. 
Therefore, simply adding an additional time stamp buff- 
er for each decoder and assuming that each access unit 10 
has an accompanied time stamp is not possible. Since 
a packet header contains only one time stamp, and 
since the packet data can contain several access unit 
headers, not every access unit header will have a time 
stamp. Even if the packet data contains an access unit is 
header, inserting a time stamp in the preceding packet 
header is optional because the only MPEG requirement 
concerning time stamps is that they occur at least once 
every 07 seconds. 

[001 2] If a particular application uses two or more de- 
coders (e.g. one video and one audio decoder) it is often 
necessary to synchronize these decoders. According to 
the MPEG 1 system standard time stamps must be used 
to perform this synchronization. The standard assumes 
that the audio and video decoder processing delays are 
0 ms. Consequently (in this model) the time when an 
access unit is decoded is equal to the time when the 
decoded access unit (i.e. "presentation unit") is output. 
[0013] Besides the previously mentioned reason for 
the decoder start-up delay (to minimize initial buffer er- 
ror), in case of multi-decoder systems, for each decoder 
an initial start-up delay is also necessary. This is be- 
cause, for example, the audio and video packets that 
occur together (in the same pack) are often segments 
of the audio and video signal, that after decoding, should 
be output at different times. This is possible, because 
MPEG has agreed on a certain amount of buffering be- 
fore each video and audio decoder, which allows a flex- 
ible multiplex bitstream structure. A different way to de- 
scribe the MPEG multiplex standard is: any kind of pack 
and packet structure is acceptable, as long as the buff- 
ers in the reference decoder do not overflow or under- 
flow. 

[0014] Two kinds of decoding systems exist for syn- 
chronization. The first is a locked system wherein the 
frame rate and sampling rate are locked to a single com- 
mon clock (e.g. 90 KHz in MPEG). The locked system 
has the disadvantage that it can only play back bit 
streams that were generated by an encoding system 
where the frame rate and sampling rate were also 
locked to one common clock. Whether the video and au- 
dio encoder clocks are locked or not will depend on the 
application. (In case of CDI-FMV locking the encoders 
is mandatory). In this system, (if the transmission error 
characteristics are limited) after reading the first video 
and audio time stamps and using them for the respective 
decoder start, all following time stamps can be ignored. 
This kind of decoding system is relatively simple, and 



does not need to keep track of which time stamps belong 
to which access units. However, if a transmission error 
causes missing or false access unit headers, a sync er- 
ror results (and a corresponding buffer error). Such 
problems can also occur in non-MPEG systems. The in- 
vention provides a solution for this problem. 
[0015] The second kind of decoding system (called 
non-locked decoding system) can atso play back non- 
locked encoded multiplex bit streams. Non-locked en- 
coded bit streams are generated by encoding systems 
that have independent encoder frame and sampling rate 
clocks. In this case there is no relation between the vid- 
eo encoder's frame rate error and the audio encoder's 
sampling rate error; they vary independently. 
[0016] Non-locked MPEG decoding systems are 
used if the multiplex bitstream was generated by a non- 
locked encoding system, i.e. an encoding system where 
the picture rate clock and audio sampling rate clock are 
independent. Whether the non-locked encoded sys- 
tems will be used or not will depend on the application. 
For example, in case of CDI-FMV (i.e. Phillips' Compact 
Disc Interactive with Full Motion Video extension, which 
has adopted the MPEG 1 standard) independent video 
and audio encoder clocks are not allowed. Instead, both 
these clocks must be locked to a single common clock. 
However, in the future some applications may use non- 
locked MPEG systems for certain applications. 
[001 7] When the MPEG standard is used, non-locked 
encoder (frame and sampling rate) clock errors are re- 
corded with time stamps and then included in the bit 
stream. During playback, in order to prevent an AVsync 
error, at least one decoder must have a PLL mechanism 
which uses time stamps regularly and make the actual 
frame (or sampling) rate match the time stamp values. 
The video decoder should thus read the video time 
stamps (i.e. the DTS -- the Decoding Time Stamps) or 
the video PTS (the Presentation Time Stamps) and use 
these time stamps to control the picture rate, or the au- 
dio decoder should read the audio DTS'es and use 
these to control the audio sampling rate, or both decod- 
ers should use time stamps to control their clocks. 
[0018] In FIGURE 1, an MPEG or similar data stream 
20 of packets is shown as a mixture of video packets 
such as 22 and audio packets 24. Collections of packets 
22 and 24 are arranged in a larger pack preceded by a 
pack header 26. In each case, the actual video data 27 
or audio data 28 are preceded by a video packet header 
which contains (among other data items) time stamp 30 
or an audio packet header which contains (among other 
data items) time stamps 32 respectively. The actual vid- 
eo data 27 are divided into video frames, whereas the 
audio data 28 are divided into batches of samples as 
illustrated. 

[001 9] According to one decoding method, the decod- 
ing system demultiplexes the incoming packets into an 
audio bit stream and a video bit stream takes the time 
stamps from the packet header and inserts them just 
before the related access unit in each elementary 
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stream. This generates the syntax as shown in FIGURE 
2. In this syntax, for example, frame n+1 contains the 
value of the video time stamp (VTS) (i.e. a time value 
from a 90Khz clock). Similarly, unitm+1 includes the au- 
dio time stamp (ATS) just prior to the audio unit m+1 . s 
[0020] In order to produce such elementary streams, 
a decoding system as shown in FIGURE 3 can be used. 
In this system, the multiplexed bit stream 20 is provided 
to a demultiplexer 50 which separates the bit stream into 
video data, video time stamps, audio data and audio 
time stamps. The video data are passed through a video 
syntax modifier 54 while the audio data are passed 
through an audio syntax modifier 58. The modified video 
and audio data are buffered in buffers 60 and 62 respec- 
tively prior to decoding by video and audio decoders 66 
and 68 respectively under control of picture rate control 
circuit 74 and sample rate control 76. Video and audio 
emerge at the outputs 84 and 86 respectively. 
[0021] This method has the drawback that extra mod- 
ules are required before each buffer in order to insert 
the time stamps in the right place of the demultiplexed 
data streams. If the time stamps were in the correct lay- 
er, this problem would not exist. 
[0022] Also, this non-locked system has a further 
drawback that both the elementary A/V bit streams are 
modified (i.e. the bit streams at the input of the decoders 
do not comply with the respective audio and video 
standards). Therefore, the decoders of this system can- 
not directly decode non-multiplexed audio and video bit 
streams. 

[0023] Finally, this non-tocked system succeeds in 
maintaining the relation between time stamps and ac- 
cess unit headers. Therefore, it has the ability to detect 
whether access unit headers are lost or falsely gener- 
ated (e.g. due to a transmission error). Such an error 
would cause a large difference between the intended 
access unit decoding time (i.e. the time stamp's value) 
and the actual access unit's decoding time which would 
be detected by the affected decoder's PLL. However, 
unfortunately the PLLs are designed to correct such dif- 
ferences by adjustments of the decoder's clock, which 
in the worst case will cause a third drawback. The PLL 
may try to repair such a large AV sync error by adjusting 
the decoder rate very slowly (as usual). Such a slow cor- 
rection procedure requires a long time to repair the AV 
sync error and the corresponding buffer error. Conse- 
quently, the chance of buffer underflow or overflow in- 
creases. Also, since a large AV sync error could last for 
several seconds or longer, the chance that the user will 
notice -the AV sync error increases. If the PLL attempts 
to repair the AV error by quickly adjusting the affected 
decoder's clock, other audio or video artifacts would be 
generated (e.g. vertical roll in the videoor audiofrequen- 
cy shift in the audio). Therefore in this system the ad- 
justment of a decoder clock with time stamp values is 
appropriate for replicating the small encoder clock er- 
rors that exist in non-locked encoded composite bit 
streams. This approach is not always suitable for cor- 
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recting large sync errors that were caused by a certain 
number of lost or excess access unit headers. 
[0024] From Signal Processing Image Communica- 
tion, Vol. 3, June 1991 , No.2/3, Amsterdam, p. 157 - 165, 
LeDuc et al "Universal VBR videocodecs for ATM net- 
works in the Belgian broadband experiment" re-syn- 
chronization of a decoder clock with respect to a source 
achieved by the transmission of time stamps to lock a 
decoder clock with a PLL is known. 
[0025] From FR-A-2648972 a synchronization device 
for a variable linked decoder is known, wherein each da- 
ta unit is given a number and the numbers associated 
with the data units is compared with the numbers stored 
in registers. 

[0026] The present invention was developed to alle- 
viate these problems of non-locked MPEG decoding 
systems and to overcome the disadvantage of simple 
locked (MPEG and non-MPEG) decoding systems as 
described above. 

[0027] The present invention was designed to allevi- 
ate several problems as described above. The simplest 
MPEG decoding system (as described above) and the 
simplest non-MPEG decoding systems are locked de- 
coding systems (systems where the video decoder's 
picture rate and the audio decoder's sampling rate are 
locked to a single common clock). These systems have 
the drawback that they can not be used in applications 
that loose or falsely generate access unit headers (for 
example, due to transmission or storage errors). 
[0028] The problem is solved by means of a decoder 
and a decoding method with the features of claims 1 and 
11 respectively. 

[0029] The present invention overcomes these short- 
comings by using an access unit count value in each 
elementary stream. With this value each decoder can 
detect missing or false access unit headers. The affect- 
ed decoder can then request a system reset or try to 
"repair" the synchronization error by redecoding access 
units or by skipping access units. The decoder for such 
a system is easily implemented. In addition to this, ac- 
cess count makes a simpler non-locked decoding sys- 
tem possible without the disadvantages of the non- 
locked decoding system above. 



[0030] It is an object of the present invention to pro- 
vide an A/V decoding system which can detect missing 
or excess access unit headers. 

[0031] It is another aspect to provide an A/V decoding 
system which can maintain synchronization in an un- 
locked system. 

[0032] It is another aspect to provide an A/V decoding 
system which can play back edited multiplex and edited 
elementary bit streams. 

[0033] It is an advantage that the present invention is 
simple to implement. 

[0034] It is another advantage that the current inven- 
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tion solves the problem of determining which time stamp 
belongs to which access unit, by looking at the value of 
each time stamp and then associating it with a certain 
access unit. 

[0035] These and other objects, advantages and f ea- s 
tures of the invention will become apparent to those 
skilled in the art upon consideration of the following de- 
scription of the invention. 

[0036] A demultiplexer for a multiplexed audio video 
signal includes a clock which produces atime reference. 
A demultiplexing switch separates a composite data 
stream into audio sample batches, audio time stamps, 
video frames, and video time stamps, where the audio 
time stamps are fewer than or equal in number to the 
audio sample batches and where the video time stamps 
are fewer than or equal in number to the video frames. 
An audio phase-locked loop receives the audio time 
stamps and the time reference, and compares the time 
reference with the audio time stamps to produce an au- 
diotiming signal which associates one of the audio sam- 
ple batches with each of the audio time stamps. An au- 
dio decoder receives the audio sample batches and the 
audio timing signal, and decodes the audio sample 
batches in synchronization with the audio timing signal. 
A video phase-locked loop receives the video time 
stamps and the time reference and compares the time 
reference with the video time stamps to produce a video 
timing signal which associates one of the video frames 
with each of the video time stamps. A video decoder re- 
ceives the video frames and the video timing signal, and 
decodes the video frames in synchronization with the 
video timing signal. 

[0037] A demultiplexer for an MPEG standard multi- 
plexed audio video signal includes a 90Khz clock pro- 
ducing a time reference indicative of an absolute time 
elapsed since a reference time. A demultiplexing switch 
separates a composite data stream into audio sample 
batches, audio time stamps, video frames, and video 
time stamps, where the audio time stamps are fewer 
than or equal in number to the audio sample batches 
and the video time stamps are fewer than or equal in 
number to the video frames. A video timestamp buffer 
receives the video timestamps from the demultiplexing 
switch and an audio timestamp buffer receives the audio 
timestamps from the demultiplexing switch. A video 
buffer receives the video frames from the demultiplexing 
switch, and an audio buffer receives the audio sample 
batches from the demultiplexing switch. An audio 
phase-locked loop receives the audio time stamps from 
the audio timestamp buffer and receives the time refer- 
ence. The audio PLL compares the time reference with 
the audio time stamps to produce an audio timing signal 
which associates one of the audio sample batches with 
each of the audio time stamps. The audio phase-locked 
loop associates an audio time stamp with the audio sam- 
ple batch which is decoded closest in time, as measured 
by the clock, to the value of the audio time stamp. An 
audio decoder receives the audio sample batches and 
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the audio timing signal, and decodes the audio sample 
batches in synchronization with the audio timing signal. 
A video phase-locked loop, receives the video time 
stamps from the video time stamp buffer and receives 
the time reference. The video PLL compares the time 
reference with the video time stamps to produce a video 
timing signal which associates one of the video frames 
with each of the video time stamps, wherein the video 
phase-locked loop associates a video time stamp with 
the video frame which is decoded closest in time, as 
measured by the clock, to the value of the video time 
stamp. A video decoder receives the video frames and 
the video timing signal, and decodes the video frames 
in synchronization with the video timing signal. 
[0038] A decoder includes a clock producing a time 
reference. The composite data stream is separated into 
data units and data time stamps, wherein the data time 
stamps are fewer than or equal in number to the data 
units. A phase-locked loop, receives the time stamps 
and the time reference, and compares the time refer- 
ence with the time stamps to produce a timing signal 
which associates one of the data units with each of the 
time stamps, wherein the phase-locked loop associates 
a time stamp with the data unit which is decoded closest 
in time, as measured by the clock, to the value of the 
video time stamp. A decoder receives the data units and 
the audio timing signal, and decodes the units in syn- 
chronization with the timing signal. 
[0039] A demultiplexer for a multiplexed audio video 
signal includes a demultiplexing switch for separating a 
composite data stream into audio sample batches, au- 
dio time stamps, video frames, and video time stamps. 
An audio phase-locked loop, receives the audio time 
stamps, and produces an audio timing signal in synchro- 
nization with the audio time stamps. An audio decoder 
receives the audio sample batches and the audio timing 
signal, and decodes the audio sample batches in syn- 
chronization with the audio timing signal. The audio de- 
coder extracts an access unit count from the audio sam- 
ple batches and compares the access unit count with a 
stored access unit count to verify that the audio decoder 
is operating in proper synchronization. A video phase - 
locked loop, receives the video time stamps, and pro- 
duces a video timing signal in synchronization with the 
video time stamps. A video decoder receives the video 
frames and the video timing signal, and decodes the vid- 
eo frames in synchronization with the video timing sig- 
nal. The video decoder extracts an access unit count 
from the video frames and comparing the access unit 
count with a stored access unit count to verify that the 
video decoder is operating in proper synchronization. 
[0040] A decoder includes a demultiplexing circuit for 
separating a composite data stream into data units and 
data time stamps. A timing circuit receives the data time 
stamps, and produces a timing signal in synchronization 
with the time stamps. A decoder receives the data units 
and the timing signal, and decodes the data units in syn- 
chronization with the timing signal. The decoder in- 
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eludes a circuit for extracting an access unit count from 
the each data unit and compares the access unit count 
with a stored access unit count to verify that the decoder 
is operating in proper synchronization. 
[0041] A method for demultiplexing a multiplexed au- 
dio video signal includes the steps of: separating a com- 
posite data stream into audio sample batches, audio 
time stamps, video frames, and video time stamps; pro- 
ducing an audio timing signal in synchronization with the 
audio time stamps; decoding the audio sample batches 
in synchronization with the audio timing signal; extract- 
ing an access unit count from the audio sample batches 
and comparing the access unit count with a stored ac- 
cess unit count to verify audio synchronization; produc- 
ing a video timing signal in synchronization with the vid- 
eo time stamps: decoding the video frames in synchro- 
nization with the video timing signal; extracting an ac- 
cess unit count from the video frames and comparing 
the access unit count with a stored access unit count to 
verify video synchronization; incrementing the stored 
access unit count each time a new access unit is re- 
ceived. 

[0042] Another method for processing a video and an 
associated audio signal includes the steps of: providing 
a signal having audio sample batches and video frames; 
decoding the audio sample batches; extracting an ac- 
cess unit count from the audio sample batches and com- 
paring the access unit count with a stored access unit 
count to verify audio synchronization; decoding the vid- 
eo frames; extracting an access unit count from the vid- 
eo frames and comparing the access unit count with a 
stored access unit count to verify video synchronization; 
incrementing the stored access unit count each time a 
new access unit is received. 

[0043] Another method for processing a stream of da- 
ta bits includes the steps of: initializing a register with 
an access unit count corresponding to a first frame 
header; receiving and decoding one or more subse- 
quent frames; incrementing the register for each the 
subsequent frame and comparing the content of the reg- 
ister with an access unit count with each the subsequent 
frame; determining if the access unit count from each 
the subsequent frame is the same as that stored in the 
register to verify synchronization; omitting a video frame 
if said access unit count for said video frame is smaller 
than said stored access unit count: omitting an audio 
sample batch if said access unit count for said audio 
sample batch is smaller than said stored access unit 
count; detecting a discontinuity bit; reinitializing the reg- 
ister upon detection of the discontinuity bit. 
[0044] According to ISO's MPEG 1 system standard 
(which is intended for digital AV storage media applica- 
tions or digital AV transmission applications) in order to 
detect and correct AV synchronization errors at the de- 
coder, "time stamps" are included in the packet headers 
of the multiplex syntax. A time stamp indicates the pres- 
entation time of the first access unit header (i.e. frame 
header or audio unit header) that occurs in that packet. 



After demultiplexing time stamps are normally separat- 
ed from their related access units. This makes them dif- 
ficult to use. In, for example, non-locked decoding sys- 
tems some method must be used to find out which time 
5 stamps belong to which access units. With one embod- 
iment of the current invention a relatively simple decod- 
ing system makes use of redundancy of the time stamp 
values to determine this relation. The current invention 
solves the problem of determining which time stamps 
10 belong to which access unit, by looking at the value of 
each time stamp and then associating it with a certain 
access unit. This enables easy non-locked decoding. 
[0045] A compressed video signal and a compressed- 
audio signal decoding apparatus of an embodiment of 
*s the invention includes a local access unit counter intend- 
ed for digital AV applications where access unit headers 
(e.g. frame headers, audio unit headers or other kind of 
data unit headers) may be lost or generated by storage 
media or transmission errors. In order to prevent a re- 
sulting AV sync error a value called "access unit count- 
is included in each elementary stream. With this value 
each decoder can detect lost or false access unit head- 
ers, and correct the AV synchronization. In order to sup- 
port editing, an extra bit can be added to indicate dis- 
continuity. 

[0046] In embodiments where the access unit count 
fields are included in the syntax of each elementary bit 
stream by each encoder, there is no need to move this 
data from the multiplex layer to each elementary in the 
decoding system. Therefore the decoding system's 
hardware or software complexity is low. In this decoding 
system time stamps are used for the initial start up delay 
and synchronization, and also to follow the encoder's 
frame and/or audio sampling rate clock error. Access 
Unit Count is used only to detect missing or fake access 
unit headers. Also easier MPEG locked decoder sys- 
tems are possible, and the invention can be used in non- 
MPEG applications. 

[0047] The invention, both as to organization and 
method of operation, together with further objects and 
advantages thereof, may be best understood by refer- 
ence to the following description taken in conjunction 
with the accompanying drawing. 

Brief Description of the Drawing 

[0048] FIGURE 1 is a diagram of an MPEG data 
stream. 

[0049] FIGURE 2 is a diagram of a decoded MPEG 
data stream using one type of decoder. 
[0050] FIGURE 3 is a block diagram of a system pro- 
ducing the data stream of FIGURE 2. 
[0051] FIGURE 4 shows a diagram of an unlocked de- 
coder system of a related invention. 
[0052] FIGURE 5 shows a timing diagram of the op- 
eration of the unlocked decoder of FIGURE 4. 
[0053] FIGURE 6 shows a diagram of a data stream 
incorporating the access unit counts of the present in- 
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vention. 

[0054] FIGURE 7 is a block diagram of a decoder sys- 
tem for the data stream of FIGURE 4. 
[0055] FIGURE 8 is a timing diagram illustrating op- 
eration of the embodiment of FIGURE 7. FIGURE 9 il- 
lustrates bitstream editing. 

Detailed Description of the Invention 

[0056] While this invention is susceptible of embodi- 
ment in many different forms, there is shown in the draw- 
ings and will herein be described in detail specific em- 
bodiments, with the understanding that the present dis- 
closure is to be considered as an example of the princi- 
ples of the invention and not intended to limit the inven- 
tion to the specific embodiments shown and described. 
For example, the present invention can also be used 
with CD ROM and Interactive CD applications. Also, al- 
though the embodiments shown are for MPEG type mul- 
tiplexed systems, the invention can be used with similar 
systems. 

[0057] A partial solution to some of the problems de- 
scribed above is described in connection with FIGURES 
4 and 5 showing a decoder system according to a relat- 
ed invention. This solution may be adequate for some 
systems or with systems requiring strict conformity with 
the MPEG 1 standard. In this system, a relatively simple 
decoding system makes use of the time stamp values 
to determine the relation ship between the time stamp 
and the access unit associated therewith. In non-locked 
encoded bit streams e.g. the video time stamps indicate 
the decoding time for their respective pictures, which de- 
pends on the video encoder's picture rate (e.g. 30 Hz) 
and the video encoder's clock error. In this kind of bit 
stream, if the video encoder's picture rate clock error is 
0 and if a perfect 90KHz clock is used by the encoding 
system, the difference between consecutive time 
stamps is n*3000 where n is an integer larger or equal 
to 1 . (3000 is the number of 90KHz cycles in each picture 
period, if the picture rate is exactly 30 Hz). However with 
the clock errors, this difference can be e.g. n*3000 +1 
or -1. (The MPEG standard specifies a limit on the en- 
coder clock errors (<1 OOppm), and a limit on the interval 
between two consecutive time stamps (<0.8s)). Conse- 
quently, by looking at the value of each time stamp, and 
by counting the number of processed pictures, (a mod- 
ule of) the video decoder can find out which time stamp 
belongs to which picture and then use the time stamps 
to control the picture rate. A similar method can also be 
applied for the audio decoder if necessary. A decoder 
according to this partial solution is illustrated in FIGURE 
4. In this decoder a multiplexed bit stream 20 passes 
through a demultiplexer 50, operating under control of 
a digital signal processor (DSP) 98, separates out video 
data and sends it to video receive buffer 100. Audio data 
are sent to audio receive buffer 102. Video time stamps 
are sent to video time stamp buffer 1 04 while audio time 
stamps are sent to audio time stamp buffer 1 08. 



[0058] In this system, each PLL relates timestamps to 
a particular access unit. That is, each PLL 120 and 122 
associates a time stamp with the access unit which is 
decoded nearest the timestamp value. The time stamp 
5 is then used to control the clocking of the audio or video 
decoder. 

[0059] For initialization, the SCR from the first pack 
header is passed to an absolute time correction register 
114 which is driven by a clock 118 (90Khz in the case 
10 of MPEG 1 ). This absolute time value is used to drive a 
video phase-locked loop (PLL) 120 and an audio PLL 
122. Video and audio PLLs 120 and 122 respectively 
time video decoders 1 28 and 1 30 which respectively re- 
ceive the video and audio data from the video buffer 1 00 
and audio buffer 102. 

[0060] Operation of this decoder system is illustrated 
in FIGURE 5. At time T1 DSP 98 detects the first pack 
header, turns demultiplexer 50 to position D a M , and ini- 
tializes register 114 with SCRn. From here on, register 

20 1 1 4's content is incremented by one by the 90 KHz clock 
118. At time T2, DSP 98 detects the first video packet 
(ignores PTS) and turns demultiplexer 50 to position V. 
DTSm is stored in timestamp buffer 1 04. (When the next 
pack header is detected at time T3, the difference be- 

25 tween SCRn+1 and 114's contents can be used to con- 
trol the DSM bit rate.) 

[0061] At time T3, DSP 98 detects a video packet and 
turns demultiplexer 50 to w c" diverting the time stamp 
(DTS) to the timestamp buffer 104. DTSm+1 is stored 

30 jn timestamp buffer 104 and demultiplexer 50 is set to 
"b n to route the video data to buffer 1 00. At time T5, DSP 
98 sets demultiplexer 50 to n c°, and the next time stamp 
DTSm+2 is stored in timestamp buffer 1 04. Then demul- 
tiplexer 50 is then set to position "b" to again route data 

35 to buffer 100. 

[0062] At time T6, register 114's contents becomes 
the same as the first timestamp in buffer 104. PLL 120 
detects this condition and starts decoder 128. At time 
T7, DSP 98 detects an audio packet, so it sets demulti- 

40 piexer 50 to position V sending the time stamp to buffer 
1 08. DTSa is thus stored in buffer 1 08. At time T8, Reg- 
ister 114's contents becomes the same as DTSa (in 
108). PLL 122 detects this and starts decoder 130. PLL 
120 looks at the next timestamp (DTSm+1 ), knows that 

45 this frame has no time stamp. At time T9, PLL 120 de- 
tects the relation between DTSm+1 and frame i+2 and 
tries to speed up the PLL clock 120. 
[0063] This method works well if the transmission (or 
DSM) error characteristics are limited. However, if e.g. 

50 a transmission error destroys an access unit header (e. 
g. the frame header i+3 as illustrated in FIGURE 5), the 
decoding system would not have a reliable way to detect 
the missing frame header. In this example, at time T9 
the decoder fails to detect the damaged frame header 

55 j+3 and simply discards all following data until the next 
frame header i+4. At T10, the decoder processes frame 
i+4 one picture period too earty, which causes an AV 
sync error that is approximately equal to one picture pe- 
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riod. Also at T10, PLL 120 relates time stamp DTSm+2 
to the frame that is decoded near the time DTSm+2, 
which due to the transmission error to frame i+4. There- 
fore, again PLL 120 assumes that the frame rate is too 
slow and will try to speed up the frame rate clock 120. 
However due to the damaged frame header 1+3, the 
frame rate is actually too fast. 

[0064] (Note: the existing temporal reference field in 
each frame header of the MPEG video syntax does pro- 
vide some opportunity to detect missing or excess frame 
headers ; but unfortunately this value is reset to zero at 
the first frame of each Group Of Pictures (GOP). Con- 
sequently, the last frame of each GOP is not protected, 
and a video bitstream where each GOP has only one 
frame (e.g. all temporal_references zero) would be of- 
fered no protection at all. Another option might be to use 
the existing time_code in GOP header of the MPEG vid- 
eo syntax. However, has the drawback that error detec- 
tion can only be performed when receiving GOP head- 
ers. This has the drawback that in case of GOP's with 
many frames, the sync error and buffer error would last 
for a long time. Furthermore, the chances of buffer over- 
flow or underflow increases if the sync error is not cor- 
rected soon after it occurs. In any case, the MPEG audio 
standard is not believed to have any data item that can 
be used to detect missing or excess audio access unit 
headers.) 

[0065] FIGURES 4 and 5 show an example of a non- 
locked MPEG decoding system. In this system the 
timestamps can only be used for small adjustments of 
the frame and sampling rate clock. The effect of a trans- 
mission (or DSM) error which destroys an access unit 
header (in this case frame header i+3) is presented. 
Note that besides the AV sync error, also a large video 
buffer error is generated. Therefore in some cases, such 
a transmission error can also cause buffer overflow or 
underflow. If this kind of transmission error occurs in a 
locked decoding system, the same kind of problems (AV 
sync error and buffer error) are caused. 
[0066] This kind of decoding system can not detect 
whether access units headers are lost or falsely gener- 
ated (e.g. due to transmission errors). Therefore, the ap- 
plication of this system may be primarily in systems that 
have limited transmission error characteristics or re- 
quirements. To alleviate the above short coming (which 
also occurs in non MPEG applications) the following 
method is used according to the present invention. 
[0067] Turning nowto FIGURE 6, adata stream which 
is modified to include access unit counts according to 
the present invention is shown. In this data stream, vid- 
eo data 27 are arranged in frames shown as n, n+1 , etc. 
According to the present invention, each frame of video 
includes an access unit count 190 which precedes the 
frame. Access unit count 190 is assigned the value n+1 
for the frame shown. Similarly, each audio unit of audio 
data 28 includes an access unit count 192, shown as 
1 92a and 1 92b. The value assigned to access unit count 
192a is m+1 while the value assigned to access unit 



count 1 92b is m+2. Other arrangements for assignment 
of access unit count may also be used without departing 
from the present invention. 

[0068] Referring to FIGURE 7, a decoder for the data 
5 stream of FIGURE 6 is shown. In this decoder the mul- 
tiplexed bit stream 20 passes through a demultiplexer 
50, operating under control of a digital signal processor 
(DSP) 98, separates out video data and sends it to video 
receive buffer 100. Audio data are sent to audio receive 
10 buffer 102. Time stamps are sent to video time stamp 
buffer 104 while audio time stamps are sent to audio 
time stamp buffer 108. 

[0069] Some systems may not use all time stamps be- 
cause this might require a relatively large and expensive 

75 time stamp buffer 1 04. A small time stamp buffer is pref- 
erable which can hold only one time stamp at a time. 
Whenever this time stamp buffer is full, incoming time 
stamps are discarded. This has the advantage of sim- 
plified buffer management since there is no need for 

20 read or write pointers to the time stamp buffer. The dis- 
advantage is that PLL control is somewhat slower. Note, 
however, that time stamps are only used to replicate 
small encoder clock errors. Therefore, some additional 
deviation from the decoding time (as specified by time 

25 stamps) will only cause very small additional AC sync 
error and very small additional buffer error. When con- 
ventional crystals are used to drive the decoder clocks, 
even a delay of a clock control action of a few seconds 
would produce at most an additional AV sync error of a 

30 few 90 KHz clock cycles. Thus, since these errors are 
far below the half picture period sync error (in the case 
of 30 Hz, 1500 90 KHz clock cycles) it is still easy to 
associate time stamps with their related access units. 
[0070] As with the system of FIGURE 4, for initializa- 

35 tion, the SCR from the first packet header is passed to 
an absolute time correction circuit 114 which is driven 
by a clock 1 1 8 (90Khz in the case of MPEG 1 ). This ab- 
solute time is used to drive a video phase-locked loop 
(PLL) 1 20 and an audio PLL 1 22. Video and audio PLLs 

40 1 20 and 1 22 respectively time video decoders 228 and 
230 which respectively receive the video and audio data 
from the video buffer 100 and audio buffer 102. 
[0071] Video decoder 228 includes a video DSP 234 
which (after start-up) extracts the first read (first encoun- 

45 tered) access unit count from the video bitstream and 
stores it in a register 238 as a local access unit count. 
Similarly, audio decoder 230 includes a DSP 240 which 
extracts the first read access unit count from the audio 
bitstream and stores it in a register 244 as a local access 

50 unit count. 

[0072] The operation of this embodiment can be bet- 
ter understood with reference to the timing diagram of 
FIGURE 8. At time T1 , decoder 228 starts. DSP 234 in- 
itializes register 238 with the value i (which represents 

55 the access unit count from the first frame header). At 
time T2, DSP 234 decodes next frame, increments reg- 
ister 238 and compares it with current access unit count 
(i.e. from the currently processed frame). DSP 234 ver- 
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ifies that the access unit count is the same as that stored 
locally, thus the system is in synchronization. 
[0073] At time T3, DSP 234 increments register 238, 
compares it's value with the current access unit count 
(i+2) and determines that the result is proper. At time 
T4, DSP 234 increments register 238, compares it's val- 
ue (i+3) with access unit count (i+4), and thus detects 
that an error has occurred. Decoding of frame i+4 is 
postponed and the current displayed frame is redis- 
played. At time T5, DSP 234 resumes decoding of frame 
i+4 : the AV sync error is corrected. 
[0074] If the bitstream as illustrated in FIGURE 8 had 
not been corrupted due to the transmission error it 
would have complied with all three parts of the MPEG 
standard. However, since a frame header was de- 
stroyed, part of the video and part of the multiplex bit- 
streams are non-compliant. The received video bit- 
stream is non-compliant because there is an amount of 
non-decodable data following frame i+2, and because 
the frame i+3 is missing. The multiplex stream is non- 
compliant because the third video packet has a time 
stamp, but does not contain a detectable frame header. 
According to the systems standard, packets that do not 
have an access unit header, never have time stamps. 
The output of the encoding system, however, is a legal 
bit stream. 

[0075] This example shows how the AV sync error is 
detected, and then corrected. In this example, the miss- 
ing picture error is corrected immediately, that is, after 
one picture period. However a more intelligent video de- 
coder may decide to postpone the correction procedure 
until (for example) a still (or hardly moving) picture se- 
quence occurs. This might produce more realistic video 
output. The same can also be done in case of audio. 
Skipping or copying audio access units may sound bet- 
ter during a quiet part of the audio track. 
[0076] In order to continue playback, despite missing/ 
additional access units, the buffers should be made larg- 
er (than for non-error systems). This prevents under- 
flow, and (depending on the application) overflow. The 
problem of missing access units will most likely occur, 
for example, in applications where there is a high trans- 
mission rate error and a very low video bit rate. In this 
case there is a high chance that the error hits a frame 
header. 

[0077] If desired or necessary a "discontinuity bit" can 
be attached to the Access Unit Count field in order to 
enable playback of edited bitstreams. FIGURE 9 shows 
an example of how a bit stream is edited. In this case 
the editor decides to remove one frame. Thus, the 
frames would not have sequential access unit counts. 
The decoding system operates as described in figure 8, 
but it re-inrtializes the local access unit counter every 
time when the 'discontinuity bit" is set. The drawback of 
using this "discontinuity bit" is that missing/additional ac- 
cess units can not be detected if they occur at edited 
points in the bitstream. (In other words "discontinuity bit" 
= 1 disables the decoding system's "access unit count" 



checking mechanism for 1 access unit period). 
[0078] Thus it is apparent that in accordance with the 
present invention, an apparatus that fully satisfies the 
objectives, aims and advantages is set forth above. 

5 While the invention has been described in conjunction 
with specific embodiments, it is evident that many alter- 
natives, modifications, permutations and variations will 
become apparent to those skilled in the art in light of the 
foregoing description. Accordingly, it is intended that the 

to present invention embrace all such alternatives, modi- 
fications and variations as fall within the scope of the 
appended claims. 



*5 Claims 

1. A decoder for a multiplexed signal, comprising: 

a clock (118) producing a time reference; 

20 a means (50) for separating a composite data 

stream into data units and data time stamps, 
wherein said data time stamps are fewer than 
or equal in number to the data units; 
a phase-locked loop (120, 122), receiving said 

^5 time stamps and said time reference, for com- 

paring said time reference with said time 
stamps to output a timing signal when the ref- 
erence time equals a time stamp: 
a data decoder (128, 130) receiving said data 

30 units and said timing signal, associating one of 

the data units with each of the data time 
stamps, being that one of the data units that, in 
the absence of the data time stamp, would be 
decoded closest in time, as measured by the 

35 clock (118), to the value of the data time stamp, 

and decoding said data units in synchronization 
with said timing signal, and means (240) for ex- 
tracting an access unit count from said data 
units and comparing said access unit count with 

^0 a stored access unit count (244) to verify that 

said decoder is operating in proper synchroni- 
zation, wherein a discontinuity bit is attached to 
the data units if successive access units do not 
have sequential access unit counts and the 

4 5 stored access unit count is re-initialized every 

time the discontinuity bit is set. 

2. The decoder of claim 1 , further comprising: 

50 a video timestamp buffer (1 04) receiving video 

time stamps from said separating means (50). 

3. The decoder of claim 1 or 2, further comprising: 

55 an audio timestamp buffer (108) receiving au- 

dio time stamps from said separating means 
(50). 
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4. The decoder of anyone of the claims 1 to 3, further 
comprising: 

a video buffer (100) receiving video frames 
from said separating means (50). s 

5. The decoder of anyone of the claims 1 to 4, further 
comprising: 

an audio buffer (102) receiving audio sample 10 
batches from said separating means (50). 

6. The decoder of anyone of the preceding claims, 
wherein said time reference is indicative of an ab- 
solute time elapsed since a reference time. is 

7. The decoder of claim 6 : wherein said clock (118) 
operates at 90 KHz. 

8. The decoder of claim 1, wherein said multiplexed 20 
signal is an MPEG standard signal. 

9. The decoder of claim 6, wherein said phase-locked 
loop (122) associates an audio time stamp with the 
audio sample batch which is decoded closest in 25 
time, as measured by said clock ; to the value of said 
audio time stamp. 

10. The decoder of claim 6, wherein said video phase- 
locked loop (120) associates a video time stamp 30 
with the video frame which is decoded closest in 
time, as measured by said clock, to the value of said 
video time stamp. 

11. A method for decoding a multiplexed signal com- 35 
prising the following steps: 

producing (118) a time reference, 

separating (50) a composite data stream into 40 
data units and data time stamps, wherein said 
data time stamps are fewer than or equal in 
number to the data units, 

receiving (120, 122) said time stamps and said 45 
time reference for comparing said time refer- 
ence with said time stamps to produce a timing 
signal when the reference time equals a time 
stamp, and 

so 

receiving said data units and said timing signal 
to associate one of the data units with each of 
the data time stamps, being that one of the data 
units that, in the absence of the data time 
stamp, would be decoded closest in time, as 55 
measured by the time reference, to the value of 
the data time stamp and to decode (1 28, 1 30) 
said data units in synchronization with said tim- 



ing signal, wherein 

an access unit count is extracted from said 
data units and said access unit count is 
compared with a stored access unit count 
to verify synchronization; 

and 

the stored access unit count is increment- 
ed each time a new access unit is received, 
wherein a discontinuity bit is attached to 
the data units if successive access units do 
not have sequential access unit counts and 
the stored access unit count is re-initialized 
every time the discontinuity bit is set. 

1 2. The method of claim 1 1 , further comprising the step 
of duplicating a video frame if said access unit count 
for a video frame is larger than said stored access 
unit count. 

1 3. The method of claim 1 1 , further comprising the step 
of omitting a video frame if said access unit count 
for a video frame is smaller than said stored access 
unit count. 

1 4. The method of claim 1 1 , further comprising the step 
of omitting an audio sample batch if said access unit 
count for an audio sample batch is smaller than said 
stored access unit count. 

15. The method of claim 12, further comprising the step 
of duplicating an audio sample batch if said access 
unit count for an audio sample batch is larger than 
said stored access unit count. 



Patentanspruche 

1. Decodierer fur ein Multiplexsignal, mit: 

einem Taktgeber (118), der eine Zeitreferenz 
erzeugt; 

einer Einrichtung (50) zum Trennen eines zu- 
sammengesetzten Datenstroms in Datenein- 
heiten und Datenzeitstempel, wobei die Daten- 
zeitstempel zahlenmaBig weniger als Oder 
gleich den Dateneinheiten sind; 
einer Phasenverriegelungsschleife (120, 122), 
die die Zeitstempel und die Zeitreferenz emp- 
fangt, urn die Zeitreferenz mit den Zeitstempeln 
zu vergleichen ) urn ein Zeittaktsignal zu liefern, 
wenn die Referenzzeit gleich einem Zeitstem- 
pel ist; 

einem Datendecodierer (128, 130), der die Da- 
teneinheiten und das Zeittaktsignal empfangt, 
der eine der Dateneinheiten mit jedem der Da- 
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tenzeitstempel, die diese eine der Dateneinhei- 
ten ist, die in Abwesenheit des Datenzeitstem- 
pels zeitlich am nahesten - gemessen durch 
den Taktgeber (118) - decodiert wurde, dem 
Wert des Datenzeitstempels zuordnet und die 
Dateneinheiten synchron mit dem Zeittaktsi- 
gnal decodiert, und einer Einrichtung (240), urn 
einen Zugriffseinheit-Zahlstand von den Daten- 
einheiten zu extrahieren und den Zugriffsein- 
heit-Zahlstand mit dem gespeicherten 
AdreGeinheit-Zahlstand (244) zu vergleichen, 
um zu verifizieren, daB der Decodierer genau 
synchron arbeitet, wobei ein Unstetigkeitsbit 
den Dateneinheiten hinzugefugt ist, wenn auf- 
einanderfolgende Zugriffseinheiten nicht auf- 
einanderfolgende Zugriffseinheit-Zah Istande 
haben : und der gespeicherte Zugriffseinheit- 
Zahlstand re-initialisiert wird, jedesmal, wenn 
das Unstetigkeitsbit gesetzt ist. 

2. Decodierer nach Anspruch 1, der auGerdem auf- 
weist: 

einen Videozeitstempel-Pufferspeicher (104), 
der Videozeitstempel von der Trenneinrichtung 
(50) empfangt. 

3. Decodierer nach Anspruch 1 oder 2, der auGerdem 
aufweist: 

einen Audiozeitstempel-Pufferspeicher (108), 
der Audiozeitstempel von der Trenneinrichtung 
(50) empfangt. 

4. Decodierer einem der Anspruche 1 bis 3, der au- 
Gerdem aufweist: 

einen vldeopufferspeicher (100), der Videorah- 
men von der Trenneinrichtung (50) empfangt. 

5. Decodierer nach einem der Anspruche 1 bis 4, der 
auGerdem aufweist: 

einen Audiopufferspeicher (102), der Audioab- 
tastsatze von der Trenneinrichtung (50) emp- 
fangt. 

6. Decodierer nach einem der vorhergehenden An- 
spruche, wobei die Zeitreferenz die Absolutzeit an- 
zeigt, die seit einer Referenzzeit verstrichen ist. 

7. Decodierer nach Anspruch 6, wobei der Taktgeber 
(118) bei 90 kHz arbeitet. 

8. Decodierer nach Anspruch 1 s wobei das Multiplex- 
signal ein MPEG-Normsignal ist. 

9. Decodierer nach Anspruch 6, wobei die Phasenver- 



riegelungsschleife (122) einen Audiozeitstempel 
mit dem Audioabtastsatz, der zeitlich am nahesten 
- gemessen durch den Taktgeber - decodiert ist, 
dem Wert des Audiozeitstempeis zuordnet. 

s 

1 0. Decodierer nach Anspruch 6 ; wobei die Video-Pha- 
senverriegelungsschleife (120) einen Videozeit- 
stempel mit dem Videorahmen, der zeitlich am na- 
hesten - gemessen durch den Taktgeber - decodiert 

10 ist, dem Wert des Videozeitstempels zuordnet. 

11. Verfahren zum Decodieren eines Multiplexsignals, 
welches folgende Schritte umfaGt: 

Erzeugen (118) einer Zeitreferenz; 
Trennen (50) eines zusammengesetzten Da- 
tenstroms in Dateneinheiten und Datenzeit- 
stempel, wobei die Datenzeitstempel zeitlich 
weniger als oder gleich den Dateneinheiten 

20 sind; 

Empfangen (1 20, 1 22) der Zeitstempel und der 
Zeitreferenz, um die Zeitreferenz mit dem Zeit- 
stempeln zu vergleichen, um ein Zeittaktsignal 
zu erzeugen, wenn die Referenzzeit gleich ei- 

25 nem Zeitstempel ist, und 

Empfangen der Dateneinheiten und des Zeit- 
aktsignals, um eine der Dateneinheiten mit je- 
dem der Datenzeitstempel, die diese eine der 
Dateneinheiten ist ; die in Abwesenheit des Da- 

30 tenzeitstempels zeitlich am nahesten - gemes- 

sen durch die Zeitreferenz - decodiert wurde, 
dem Wert des Datenzeitstempels zuzuordnen 
und um die Dateneinheiten synchron mit dem 
Zeittaktsignal zu decodieren (128, 130), wobei 

35 ein Zugriffseinheit-Zahlstand von den Daten- 

einheiten extrahiert wird und der Zugriffsein- 
heit-Zahlstand mit dem gespeicherten Zugriffs- 
einheit-Zahlstand verglichen wird, um die Syn- 
chronisation zu verifizieren; und 

40 der gespeicherte Zugriffseinheit-Zahlstand 

dann inkrementiert wird, jedesmal, wenn eine 
neue Zu griff seinheit empfangen wird, wobei ein 
Unstetigkeitsbit den Dateneinheiten hinzuge- 
fugt wird, wenn aufeinanderfolgende Zugriffs- 

45 einheiten nicht aufeinanderfolgende Zugriffs- 

einheit-Zahlstande haben s und der gespeicher- 
te Zugriffseinheit-Zahlstand jedesmal dann re- 
initialisiert wird t wenn das Unstetigkeitsbit ge- 
setzt wird. 

50 

12. Verfahren nach Anspruch 11 . welches auGerdem 
den Schritt umfaGt, einen Videorahmen zu verviel- 
faltigen, wenn der Zugriffseinheit-Zahlstand fur ei- 
nen Videorahmen groGer ist als der gespeicherte 

55 Zugriffseinheit-Zahlstand. 

13. Verfahren nach Anspruch 11, welches auGerdem 
den Schritt umfaGt, einen Videorahmen auszulas- 
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sen, wenn der Zugriffseinheit-Zahlstand fur einen 
Videorahmen kleiner ist als der gespeicherte Zu- 
griffseinheit-Zahlstand. 

14. Verfahren nach Anspruch 11, welches auBerdem s 
den Schritt umfaBt, einen Audioabtastsatz auszu- 
lassen, wenn der Zugriffseinheit-Zahlstand fur ei- 
nen Audioabtastsatz kleiner ist als der gespeicherte 
Zugriffseinheit-Zahlstand. 

10 

15. Verfahren nach Anspruch 12, welches auBerdem 
den Schritt umfaBt, einen Audioabtastsatz zu ver- 
vielfaltigen, wenn der Zugriffseinheit-Zahlstand fur 
einen Audioabtastsatz groBer ist als der gespei- 
cherte Zugriffseinheit-Zahlstand. is 



Revendications 

1 . Decodeur pour un signal multiplexe, comportant : 20 

une horloge (118) produisant une reference de 
temps ; 

un moyen (50) pour separer un train de don- 
nees composite en unites de donnees et en re- 25 
peres de temps de donnees, lesdits reperes de 
temps de donnees etant moins nombreux que 
ou egaux en nombre aux unites de donnees ; 
une boucle a verrouillage de phase (120, 122), 
recevant lesdits reperes de temps et ladite re- so 
ference de temps : afin de comparer ladite refe- 
rence de temps auxdits reperes de temps pour 
delivrer un signal de synchronisation lorsque le 
temps de reference est egal a un repere de 
temps; 35 
un decodeur de donnees (128, 130) recevant 
lesdites unites de donnees et ledit signal de 
synchronisation associant une des unites de 
donnees a chacun des reperes de temps de 
donnees, celle des unites de donnees qui, en 40 
I'absence du repere de temps de donnees, se- 
rait decodee la plus proche en temps, comme 
mesure par Phorloge (118), de la valeur du re- 
pere de temps de donnees, et decodant lesdi- 
tes unites de donnees en synchronisme avec 45 
ledit signal de synchronisation, et un moyen 
(240) afin d'extraire un comptage d'unite d'ac- 
ces desdites unites de donnees et de comparer 
ledit comptage d'unite d'acces a un comptage 
d'unite d'acces memorise (244) afin de verifier 50 
que ledit decodeur fonctionne en synchronisa- 
tion correcte, un bit de discontinuite etant fixe 
aux unites de donnees si des unites d'acces 
successives n'ont pas des comptages d'unite 
d'acces sequentiels et le comptage d'unite ss 
d'acces memorise est reinitialise chaque fois 
que le bit de discontinuite est place. 



2. Decodeur selon la revendication 1 , comportant en 
outre une memoire-tampon de repere de temps vi- 
deo (1 04) recevant des reperes de temps video du- 
dit moyen de separation (50). 

3. Decodeur selon la revendication 1 ou 2, comportant 
en outre une memoire-tampon (108) de reperes de 
temps audio recevant des reperes de temps audio 
dudit moyen de separation (50). 

4. Decodeur selon I'une quelconque des revendica- 
tions 1 a 3, comportant en outre une memoire-tam- 
pon video (100) recevant des trames video dudit 
moyen de separation (50). 

5. Decodeur selon I'une quelconque des revendica- 
tions 1 a 4, comportant en outre une memoire-tam- 
pon audio (102) recevant des lots d'echantillons 
audio dudit moyen de separation (50). 

6. Decodeur selon I'une quelconque des revendica- 
tions pr6c6dentes, dans lequel ladite reference de 
temps est representative d'un temps absolu ecoule 
depuis un instant de reference. 

7. Decodeur sefon la revendication 6, dans lequel la- 
dite horloge (118) fonctionne a 90 KHz. 

8. Decodeur selon la revendication 1 , dans lequel ledit 
signal multiplexe est un signal normalise MPEG. 

9. Decodeur selon la revendication 6, dans lequel la- 
dite boucle a verrouillage de phase (122) associe 
un repere de temps audio au lot d'echantillons audio 
qui est decode le plus pres en temps, tel que me- 
sure pare ladite horloge, de la valeur dudit repere 
de temps audio. 

10. Decodeur selon la revendication 6, dans lequel la- 
dite boucle a verrouillage de phase video (120) as- 
socie un repere de temps video a la trame video qui 
est decoded la plus proche en temps, tel que me- 
sure par ladite horloge : de la valeur dudit repere de 
temps video. 

11. Procede de decodage d'un signal multiplexe, com- 
portant les etapes consistant a : 

produire (118) une reference de temps, 
separer (50) un train de donnees composite en 
unites de donnees et en reperes de temps de 
donnees, lesdits reperes de temps de donnees 
etant moins nombreux que ou egaux en nom- 
bre aux unites de donnees, 
recevoir (120, 122) lesdits reperes de temps et 
ladite reference de temps afin de comparer la- 
dite reference de temps auxdits reperes de 
temps pour produire un signal de synchronisa- 
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tion lorsque la reference de temps est egale a 
un repere de temps, et 

recevoir lesdites unites de donnees et ledit si- 
gnal de synchronisation pour associer une des 
unites de donnees a chacun des reperes de s 
temps de donnees, cede des unites de donnees 
qui, en I'absence du repere de temps de don- 
nees, serait decodee la plus proche en temps, 
tel que mesure par la reference de temps, de 
la valeur du repere de temps de donnees et 10 
pour decoder (1 28, 1 30) lesdites unites de don- 
nees en synchronisme avec ledit signal de syn- 
chronisation, 

un comptage d'unite d'acces etant extrait des- 
dites unites de donnees et ledit comptage d'uni- 1$ 
tes d'acces 6tant compare a un comptage 
d'unite d'acces memorise afin de verifier la syn- 
chronisation, et 

le comptage d'unites d'acces memorise etant 
increments chaque fois qu'une nouvelle unite 20 
d'acces est recue, un bit de discontinuity etant 
fixe aux unites de donnees si des unites d'ac- 
ces successives ne possedent pas des comp- 
tages d'unite d'acces sequentiels et le compta- 
ge d'unite d'acces memorise est reinitialise 25 
chaque fois que le bit de discontinuity est place. 

12. Procede selon la revendication 11, comportant en 
outre I'etape de duplication d'une trame video si le- 
dit comptage d'unite d'acces pour une trame video 30 
est superieur audit comptage d'unite d'acces me- 
morise. 

13. Procede selon la revendication 11, comportant en 
outre I'etape consistant a omettre une trame video 35 
si ledit comptage d'unite d'acces pour une trame vi- 
deo est inferieur audit comptage d'unite d'acces 
memorise. 

14. Procede selon la revendication 11, comportant en 40 
outre I'etape consistant a omettre un lot d'echan- 
tillons audio si ledit comptage d'unite d'acces pour 

un lot d'echantillons audio est inferieur audit comp- 
tage d'unite d'acces memorise. 

45 

15. Precede selon la revendication 12, comportant en 
outre I'etape consistant a dupliquer un lot d'echan- 
tillons audio si ledit comptage d'unite d'acces pour 
un lot d'echantillons audio est superieur audit comp- 
tage d'unite d'acces memorise. 50 
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Fig. 5 
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Fig. 8 
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